Method and apparatus for storing data on application-level activity and other user information to enable real-time multi-dimensional reporting about user of a mobile data network

ABSTRACT

A method and apparatus for storing data on application-level activity and other user information to enable real-time multi-dimensional reporting about a user of a mobile data network. A data manager receives information about application-level activity from a mobile data network and stores the information to provide dynamic real-time reporting on network usage. The data manager comprises a database, data processing module, and analytics module. The database stores the application-level data for a predetermined period of time. The data processing module monitors the data to determine if it corresponds to a set of defined reports. If the data is relevant, the processing module updates the defined reports. The analytics module accesses the database to retrieve information satisfying operator queries about network usage. If the operator chooses to convert the query into a defined report, the analytics module creates a newly defined report and populates it accordingly.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority under 35 U.S.C. §119(e) to U.S.Provisional Application Ser. No. 60/990,328, filed on Nov. 27, 2007entitled “Method and Apparatus for Providing Contextual User-LevelReal-Time Visibility Into Mobile Content Consumption,” the disclosure ofwhich is hereby incorporated by reference in their entirety.

This application is related to U.S. Provisional Application Ser. No.61/039,436, filed on Mar. 26, 2008, entitled “Method and Apparatus forReal-Time Brokering of Mobile Subscriber Information for Personalizingof Advertising and Content,” the contents of which are incorporated byreference in its entirety.

This application is related to the following applications filedconcurrently herewith, the entire contents of which are incorporatedherein by reference:

U.S. patent application Ser. No. (TBA), entitled “System and Method forCollecting, Reporting and Analyzing Data on Application-Level Activityand Other User Information on a Mobile Data Network;”

U.S. patent application Ser. No. (TBA), entitled “Method and Apparatusfor Real-Time Collection of Information About Application Level Activityand Other User Information on a Mobile Data Network;” and

U.S. patent application Ser. No. (TBA), entitled “Method and Apparatusfor Real-Time Multi-Dimensional Reporting and Analyzing of Data onApplication Level Activity and Other User Information on a Mobile DataNetwork.”

BACKGROUND OF THE INVENTION

1. Technical Field

The present invention relates generally to mobile networks andspecifically to a method and apparatus for a real-time, transparent,network-based approach for capturing multi-dimensional user-level usageinformation on usage of content and providing correlated real-timereports.

2. Discussion of Related Art

Traditionally, mobile operators have had very tight control on thecontent that was being accessed on their networks and used to limit useraccess to a ‘walled garden’ or ‘on-deck content’. This was done for tworeasons: to optimize their network for well-understood content, and tocontrol user experience. With the advent of more open devices and fasternetworks, there is an increasing trend in the mobile community to access‘off-deck’ or ‘off-portal’ content, which is content generally availableon the Internet at large and not pre-selected content hosted by theoperator. This movement is generally troubling to service providers fortwo reasons. First, service providers have very limited visibility inthe usage of off-deck content and hence they don't have the ability todesign and optimize their networks for this usage. Further, they also nolonger have the ability to control what their users access and hencethey fear becoming ‘dumb pipes’ and not participating in the wholemovement towards advertising and monetizing Internet content.

Content providers on the other hand, are interested in the potentiallyhuge increased customer base of mobile users. Further, the mobile deviceis highly personal and by getting specific information about users suchas location, demographics, usage patterns, etc. they can generate verytargeted content and advertising. However, they too lack detailedvisibility about mobile users or about what is happening in the mobilenetwork. While a client on the mobile handset could provide some ofthis, they can't put clients or other applications into mobile deviceseasily to get additional data since these devices are still fairlyrudimentary in comparison with a PC. Also, due to the traditionallock-in on the devices form a mobile operator, the client on the devicemay not provide all the detailed information. For instance, userlocation is not easily exposed by carriers since they are concernedabout privacy and its usage and also since its such a critical part ofthe carrier data. Recent developments such as the Android open platformfrom Google are attempts to open up some of this information. However,it is still up to the carriers to allow these devices on their networksand for device manufacturers to use this platform. Further, thisrestricts the ability of data collection only to the new devices thatembrace this platform—a carriers network will continue to have manyother devices as well.

A key requirement to enable these two silos—mobile carriers and contentproviders—to jointly evolve the mobile content ecosystem is to mine andshare mobile content usage effectively. By getting visibility intooff-deck mobile content usage, mobile operators can optimize theirnetworks. Mobile carriers are sitting on a goldmine of data thatincludes user's location, access patterns, demographic information, etc.By systematically sharing information between mobile operators andcontent providers, it is possible to offer very targeted and relevantcontent to the users.

Existing methods do not provide a method to capture user informationtransparently across multiple dimensions in real-time. The existingapproaches can be characterized by several categories of solutions.

The first approach used by network based Deep Packet Inspection vendors(e.g. Ellacoya) is to capture information only around a couple ofdimensions, including application and bandwidth. For example, it helpsanswer questions such as—what fraction of users are running whatapplication, or what fraction of bandwidth is used by what application.Also, these approaches don't allow for storage and analytics on thedata.

The second approach used by event monitoring products such as Vallentput probes in the network to capture events generated by other networkelements—they do not actually capture live user-level session data or doany correlation.

A third approach by instrumentation companies like Keynote focuses oncapturing on-demand measurements through their own devices emulatingreal-world testing. Their focus is on how applications run on differenthandsets, on different networks. They allow content providers andcarriers to test mobile applications on 1000s of device profiles to makesure the application runs as expected. This is accomplished throughvirtual handsets deployed on the network. They also carry actual devicesthat can be to test applications. This “sampling” approach does not givespecific user-level information that can be used for delivering a‘relevant’ mobile experience.

A fourth approach by vendors such as Telephia/Nielson andComscore/M:Metrics focuses on generated syndicated consumer research andpanel based solutions. Their approach involves placing distributedmonitors and collecting aggregated data for marketing and consumer usagecharacterization—they do no capture per-user data. Another approach isto place agents on the device to gather information. In either case,they don't collect from the network level and is hence restricted tocollecting only representative data from field monitors.

A fifth approach used by traditional Web Analytics vendors (e.g.Onmiture) relates to using logs on the application (e.g. HTTP). Thetraditional web approach does not work well for mobile applications fora number of reasons. First, this is restricted to a single application,which is HTTP. Mobile analytics requires a view across applications suchas SMS, WAP, Downloads, etc. Further, these applications don'tnecessarily have logs and also logs tend to be time-delayed. Also, theweb analytics tend to rely on client side support such as javascript,cookies, etc. which are not expected to be available universally onmobile devices. Unlike Web techniques, metrics such as unique useridentifiers and location can't be derived for mobile devices fromcookies or IP addresses. Cookies are not supported on mobile devices andIP addresses often tend to be masked when leaving the mobile network.Also, traditional web approaches to determine location through reverseIP mapping or other techniques don't apply since IP addresses aremasked.

SUMMARY

The present invention provides a method and apparatus for storing dataon application-level activity and other user information to enablereal-time multi-dimensional reporting about a user of a mobile datanetwork.

In one aspect of the invention, a data manager receives informationabout application-level activity from a mobile data network and storesthe information so as to provide dynamic real-time reporting on usage ofthe network. The data manager comprises a database, data processingmodule, and analytics module. The database stores the application-leveldata for a predetermined period of time. The data processing modulemonitors the data to determine if it corresponds to a set of definedreports. If the data does correspond to the defined reports, the dataprocessing module updates the defined reports accordingly. The analyticsmodule accesses the database to retrieve information satisfying operatorqueries about network usage. If the operator chooses to convert thequery into a defined report, the analytics module creates a newlydefined report and populates it accordingly.

In another aspect of the invention, each defined report has acorresponding table that specifies the data to be automatically retainedand updated.

In another aspect of the invention, the reports specify aggregateuser-level data, including: the physical location of the users;demographic information about the users; content site accessed by theusers; media type used by the users; device type used by the users; andnetwork type used by the users.

In another aspect of the invention, the data manager uses IPaddress-mobile phone number pairings to query carrier information aboutthe users, and augments the application-level data with the carrierinformation.

In another aspect of the invention, the analytics module compares anoperator query to the defined tables to determine whether any of thetables contain data to satisfy the query. If no relevant tables exist,the analytics module creates a new table to satisfy the operator queryand modifies the set of defined reports to include the newly createdreport.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present invention and theadvantages thereof, reference is now made to the following descriptionstaken in conjunction with the accompanying drawings, in which:

FIG. 1 shows the different elements in a mobile data network;

FIG. 2 shows the different dimensions of mobile data usage which certainembodiments of the invention may report about;

FIG. 3 shows the overall operation of the monitoring platform of certainembodiments;

FIG. 4 shows different components of the monitoring platform of certainembodiments;

FIG. 5 shows the architecture for the monitoring platform of certainembodiments;

FIG. 6 shows different approaches to deploy the collector in a networknon-intrusively;

FIG. 7 shows two specific architectures for collector deployment;

FIG. 8 shows a block diagram of the collector architecture;

FIG. 9 shows the operational flow in the collector of certainembodiments;

FIG. 10 shows mobile number to IP address correlation within thecollector of certain embodiments;

FIG. 11 shows message sequences that may be used to identifyadvertisements according to certain embodiments;

FIG. 12 shows the management architecture between collectors and areports manager of certain embodiments;

FIG. 13 shows the block diagram of the data manager of certainembodiments;

FIG. 14 shows the internal architecture of a data manager of certainembodiments;

FIG. 15 shows operational flow of a data processor of certainembodiments;

FIG. 16 shows a data schema of certain embodiments;

FIG. 17 shows the components of a reports manager of certainembodiments;

FIG. 18 shows the operational view of the reports manager of certainembodiments;

FIG. 19 specifies operational details of the reports manager of certainembodiments;

FIG. 20 shows the user interface components of the reports manager ofcertain embodiments;

FIG. 21 shows the drill down functionality of the reports manager ofcertain embodiments;

FIG. 22 shows a user-level drill down.

FIG. 23 shows how aggregate tables and defined reports are designed inthe data manager.

DETAILED DESCRIPTION

Preferred embodiments of the invention provide a monitoring platformthat provides monitoring and analytics in mobile data networks. Theanalytics are related to a number of dimensions that characterize such asystem. The system and method monitor application level (e.g., layer 7)events and activity transparently and in real-time, and allow theactivity to either be aggregated or correlated with specific user (e.g.,phone number). This method and apparatus may be used to effectively minea carrier's network to collect user-level mobile data usage informationin order to provide in-depth real-time visibility into usage for bothcarriers and content providers.

The system and method are network-based, non-intrusive, transparent, andclient-less. The data includes but is not limited to the user'sidentity, demographic information, physical location, content accessed,media type, content classification, usage patterns, device type, networkspeed, etc. The system and method may correlate these multipledimensions to generate a single anonomized description of the contentsession, in real time. The collected information can be used to feedadvertisers in real-time for targeted advertising and also for gettingaggregated data for optimizing mobile networks and applications.

The design allows collection of different types of data applications,including but not limited to HTTP, WAP, SMS, Video, Audio, Radio,Streaming, Gaming, and other applications. The design also allowscorrelation of the users' IP address with the mobile number, therebygetting access to the user's demographic information associated withmobile number. Moreover, the design allows using the phone number to getan approximation of the user's home location. Under one embodiment, thegenerated reports comprise aggregate data, without exposinguser-specific information.

Among other things, the design allows the data to be used for:

-   -   (a) Providing detailed visibility to the service provider so        that the service provider can optimize their network and improve        user experience and reduce costs    -   (b) Providing detailed visibility to the service provider so        that the service provider can determine appropriate content        relationships to increase usage on their network    -   (c) Providing detailed visibility to content providers so that        content providers can tailor and adapt their content and        delivery accordingly    -   (d) Providing detailed real-time visibility to advertisers so        that they can insert appropriate targeted advertisements into        the user's content. This can be used to individualize the user        experience with the most relevant information and thereby also        improve the monetization of the content.

FIG. 1 shows the overall context of the preferred environment of thisinvention in a mobile data network. A generic architecture is shown,which could map to either GSM or CDMA technologies. Mobile devicesconnect through the base stations and the mobile data core through arouter Gateway GPRS Serving Node (GGSN), Packet Data Serving Node(PDSN), Home Agent (HA) GGSN/PDSN/HA 110. The GGSN is in a GSM network.In a CDMA network, the devices connect through a PDSN/HA. In case thenetwork is based on simple IP, there may not be a HA but just a PDSN.The mobile data request may be sent to content and application serversoutside the mobile network 120 (this is often referred to in theindustry as “off deck” or “off net”) or to an operator portal 140 via aWAP gateway 130. The data request may also be to application servers 150which may be internal or external to the operator. The data at theoutput of the GGSN 110 thus comprises all types of data applications,including Web, WAP, video, audio, messaging, downloads, and othertraffic. In addition, the mobile data network has a subscriber database160 that manages subscriber information. This could also be a CustomerRelationship Management (CRM) database or an Authentication,Authorization, and Accounting (AAA) server. The network also consists ofa location platform 170. Other types of data sources could be ShortMessaging Service Center (SMSC) 180 that manages messaging traffic. TheBilling System 190 manages the billing information for user services.

FIG. 2 shows the details of the different dimensions across which userdata can be characterized. For a specific user IP address retrieved fromthe session request from the device, the monitoring platform collectsuser information across multiple “dimensions”, including Location 210,Demographics 220, Content type and classification 230, Application orMedia type 240, Device type 250, and network type 260. Further, historicinformation about a user can be collected to develop user profiles.Details of these dimensions are given next. Each dimension may haveseveral attributes, as is shown below each dimension.

1. Physical Location (Region, State, Cell Site, detailedlatitude/longitude level): The user's current physical location iscollected. Depending on the level of granularity available, the locationcould be at a cell-site level or right at the lat/long level. Thisinformation is collected from the Home Location Register (HLR) or aLocation Based Services (LBS) platform by indexing for the user's mobilenumber. This information is used to provide detailed targeting ofcontent and advertisements to the user. This information is the mostprecious piece of information that is only available by virtue of theuser being on a mobile network. For instance, in the case of a DSL orcable modem, it is possible to retrieve the user's physical location orhome address—but it is static and limits the amount of advertisingpossible. By getting the mobile location, it is possible to tailor thecontent and advertising to the user's current location. Note that if theuser's accurate location is not available, an approximation can beobtained by using the user's phone number. The phone number gives theuser's ‘home location’. This helps figure out for a particular session,what general area is the user's ‘home area’. Further, sometimes the SGSNor PDSN identifier is also provided in the RADIUS message. This can beused to get an approximation of the user's location.

2. Demographic information: The user's demographic information includeselements like: Address, Gender, Income, Age, Marital Status, Education,Occupation, etc. This information can be collected from differentsources, including a AAA database, a CRM system, or the HLR. Otherinformation includes the user's rate plan. Similar to location, thisinformation is used to target content and advertisements to the user.Just like location, demographic information is also uniquely availableby being on a mobile network. The reason is that the user's mobile phoneis very personal and can most often be associated with a specificindividual, unlike a PC. Further, the mobile operator has very detailedinformation on the user as part of providing the subscription. As aresult, demographic information is one of the key pieces of data that isobtained from being on a mobile network. Correlation of the demographicinformation with current location opens up a whole new dimension ofpersonalization. Not that the demographics information in itself is notenough to characterize a user, especially given the ‘family’ plans. TheCRM data often stores information related to the primary account holder.By tying in the other dimensions, a better profile of the user may bebuilt.

3. Content site accessed (destination address) & classification: Thiscaptures the specific destination requested for this session, e.g.mobile.youtube.com and is obtained by parsing the session request.Content classification (details on content,sports/shopping/car-shopping, etc.) captures details on the type ofcontent, which includes specific classification. For instance, shopping,news, games, social networking, movies, etc. This information could beinferred from the high-level content request. It is also possible to getmore specific information from a content provider who can provide thehigh-level indexing of their content—since they probably already do thisfor advertising and internal reporting. This information is useful forspecific advertisement targeting.

4. Media type (web, video, audio, gaming): This captures the type ofapplication the user is accessing—web/WAP, video, audio, gaming, Peer toPeer, etc. Further, details such as the protocol used—RTSP, HTTP,MMS—can be captured as can details on the application requesting it(Flash, Real Player, etc.). This information is also available fromdetails in the HTTP request.

5. Device type (capabilities of device, available applications): Thetype of device that was used to make the request is also retrieved. Thisis useful to understand capabilities of the device so content can beformatted appropriately. Typically the User Agent or UA characteristicscan be derived from the request.

6. Network speed (capability of network the access is coming from): Thenetwork speed can be obtained by querying the mobile network databasessuch as the HLR for whether the user is on a 3G or 2.5G connection. Insome cases this information can also be captured from the sessionrequest, depending on the protocol.

7. Usage history (historical usage patterns): The user's usage historyis derived from local data in the Umber platform based on historicaltracking. It includes types of sites visited, types of locationsvisited, or time of day behavior. While a specific content provider sitehas this information, what is unique about the monitoring platform isthat it is now available across all content sites the user is accessing,without requiring each site to provide this information. Thisinformation can also be very useful for advertising across the user'sinterests.

FIG. 3 describes the overall operation of the distributed platform(Monitoring Platform).

Step 310 is the process of collecting data off the network. Thisincludes collecting data at the output of the GGSN/HA 110 as shown inFIG. 1. Details of this step are provided further in FIG. 8.

Step 315 is the process of filtering out unnecessary data, such asmanagement traffic on the wire, the actual body of the response, images,etc. A black list is defined to define the data that needs to befiltered out. In some cases an additional white list may be needed todefine ‘allowed’ traffic. For example, all images may be on the blacklist, which images from specific ‘advertisement’ sources may be capturedif one needs to capture ads.

Step 320 is the step of collecting user ID information. Note that the IDon the output of the GGSN/HA is typically an IP address. Information inthe mobile network is typically identified through a mobile number. Step320 relates to capturing the phone number. The mobile number is derivedfrom a AAA database or by transparently monitoring (called “sniffing”)IP address assignment requests from a RADIUS server. Further details onhow this is accomplished are provided later in the document.

Step 325 relates to correlating the mobile number and IP address. Notethat IP address assignments are typically dynamic and transient. Detailson the correlation are described in FIG. 10.

Step 330 relates to transferring the filtered and correlated data to acentralized location. The frequency of the transfer can be controlled.

Step 340 relates to combining data from multiple collection sources suchas step 330 above. In addition, this step would also involve gettingdata from additional sources such as logs, client generated data, feedsfrom other operator systems such as Subscriber data 160 in FIG. 1. Othersources might also be the SMSC 180 in FIG. 1. At the end of this step,all the relevant data ‘events’ from the data network have been captured.

Step 350 relates to augmenting the events with additional information.This includes enriching specific data from other feeds. For instance,the user's phone number may be used to look up the demographicinformation (e.g., by querying profiles and/or databases maintained bythe carrier provider of the mobile data network), which can be added tothe event data. In addition, metrics are computed on each event. Metricsinclude time spent and data consumed. The time spent metric allowscomputing how much time was spent on this specific event while dataconsumed captures the amount of data used in this event. Computing thesemetrics requires session state analysis whereby events belonging to thesame session are identified. Events belonging to the same session areidentified through the IP address. The time spent on a complete sessionas well as during sub-sessions can be computed. In addition, othermetrics such as bandwidth consumed can be computed by monitoring all thesessions to and from a specific user session. The details of this stepare provided in FIG. 14.

Step 355 involves storing this enriched ‘event’ data into the database.

Step 360 involves the database computing the reports results andaggregates as the data comes along. As a result, the reports areavailable in near-real time, unlike classic data warehouses that takehours if not days to process queries.

Step 365 involves storing the reports in a database.

Step 370 involves the step of the user of the system (e.g. a personwithin the service provider or content provider) using a UI to accessthe reports.

Step 375 involves the steps of the user requesting and the systempresenting the pre-defined reports.

Step 380 involves the user specifying an ad-hoc query through agraphical user language.

Step 385 involves the system converting a selected ad-hoc query into apre-defined report so in future it gets computed automatically as otherpre-defined reports in Step 375.

FIG. 4 shows an exemplary arrangement of collector devices in a mobiledata network along with data managers and reports managers. Themonitoring platform system comprises 3 elements:

-   -   1. Collector 410: This is the network element that collects data        off the physical network segment, performing the functions 310,        315, 320, 325, and 330 in FIG. 3. Collectors 410 can be deployed        at the output of the GGSN/PDSN/HA 110 of FIG. 1. In addition,        collectors can also be deployed at a SMSC 420 or to collect data        from clients as shown in 430. A typical deployment will have        multiple collectors, typically one per GGSN/PDSN/HA. Additional        details on the collector are provided in FIGS. 6-12;    -   2. Data Manager 440: This is the centralized store getting data        from multiple collectors. This also performs the enrichment        function from step 350 in FIG. 3. In addition, this includes the        database and the associated report generation logic in steps 355        and 360 in FIG. 3;    -   3. Reports Manager 450: The Reports Manager provides the user        interface and associated logic to provide the reports. The        Reports Manager is accessed by people within a service provider        or content provider through a thin client interface. The Reports        Manager performs the functions 370, 375, and 380 in FIG. 3. The        logic for step 385 in FIG. 3 is potentially split between the        Reports Manager and the Data Manager.

In a typical deployment all the elements will reside within the serviceprovider network. Specifically, the servers for the collector, DataManager, and Reports Manager may be deployed in an operator datacenter.Collectors will typically be physically co-located with GGSN or PDSN/HAservers. The Data Manager and Reports Manager may be deployed in acentral data center. Other deployment architectures are possible wherethe Data Manager and Reports Manager may be externally deployed outsidethe operator network or hosted by a third party data center. While theData Manager and Reports Manager are shown as two physical in FIG. 4,there is no restriction on this physical implementation. For instance,the two functions may be combined into a single physical server in adeployment. Alternately, the Data Manager may be split into two physicalservers where one does the Step 350 in FIG. 3 and the other does Step355 and 360. Other implementations are also possible.

FIG. 5 shows the platform architecture for the monitoring platform. Theplatform comprises 4 layers:

Data Collection and Correlation layer 510 includes the collectionfunctionality. This layer is designed so that multiple collectionelements can be applied. While in preferred embodiments a network basedcollection out of the GGSN/PDSN 512 is used, other sources of datainclude SMS 514, client software on handsets 516, location,demographics, and other feeds 518. The architecture is flexible so thatother feeds 519 such as application logs, feeds from data warehouses,other collection devices, etc. can feed into the same platform.

Data Management and Analytics layer 520 includes the pre-defined reportsdata 524 that is continuously available as well as the ability tosupport ad hoc analysis 522 as well the ability to convert 522 to 524.Other analytics can be built into the same platform. The data collectedcan also be used to build user profiles 526 as shown where user-levelinformation may be collected.

Reporting layer 530: This includes the UI and the policy language toquery the data.

In addition, layer 540 allows sharing the user-level informationcollected in 526 to be brokered with other applications such asadvertisement targeting or marketing. Each of the elements are nextdescribed in more detail.

FIG. 6 shows how a collector may be deployed in a typical operatornetwork. Note that the techniques apply to both GSM and CDMA networks.The collector monitors the output of the GGSN/PDSN/HA 110 of FIG. 1before it goes through any subsequent gateways. As a result it sees allthe data traffic coming out of the mobile data network across allapplications. Further, at this point in the network, the IP addressesare not masked by upstream proxies or gateways. The collector isdesigned to be deployed non-intrusively. Three possible, but notlimiting, approaches are described. The first approach 610 involves aphysical tap that splits the traffic to a second wire that allows thecollector to sniff the data. The second approach 620 involves a Layer 7switch or a port mirror on an existing router that can be directed topass traffic to the collector. The third approach 630 shows that theGGSN itself may have the ability to mirror traffic or provide a feed tothe collector. Many GGSN/PDSN/HA platforms have the ability to mirrortraffic like this for applications such as wire tapping or fordebugging. Further, it may also be possible to get data from a specificmodule within a GGSN by configuring the GGSN appropriately. Thisapproach may be required if the GGSN masks or encrypts the data beforesending it out on the physical network.

FIG. 7 shows additional details for the collector deployment. FIG. 7Ashows the collector before other gateways. FIG. 7B shows a preferredembodiment where the data is picked off a layer 2 tap. FIG. 7C shows howthe collector can also collect RADIUS traffic off the physical network710 or by configuring the GGSN/PDSN to send a RADIUS feed to thecollector 720.

FIG. 8 shows the internal architecture of the collector. The input tothe collector 810 shows data coming in on a network interface. Thenetwork capture card 820 gets the relevant data off the wire withappropriate filtering. Note that it is not required to get monitoring oroperations traffic: e.g. ARP, SNMP, etc. packets can be filtered beforethe data is read into the system. Note that the system may have a datacapture card or just use a regular NIC. The output of 820 is fed to oneor more protocol modules, depending on the type of data. Examples ofprotocol modules include HTTP, WAP, RTSP, SMPP, etc. The RADIUS moduleis an optional module that may get RADIUS traffic in one of the twoapproaches described in FIG. 7C. Data from the protocol modules ismanaged in internal memory 850. The correlator 860 maps the IP addressto the mobile number. Details are provided in FIG. 10. The correlatoroutput 870 is then stored in a local memory until the data is ready tobe transferred to the Data Manager 895 via the Transfer module 880. Inaddition, the IP address/Mobile number mapping may also be used tobroker real-time profiles for ad targeting. For example, for each IPaddress, the mobile number and associated profile data may be brokeredto a different server. This brokering may be done out of band orin-line. As a result, a third party application or content provider canget details about an IP address for providing targeted content oradvertisements.

FIG. 9 shows the internal operation flow for the collector.

1. The session capture module 910 continuously captures the sessioninformation for each new IP address in the system and sends it to theinternal data store 930. The capture module 910 represents the functionsperformed by modules 820 and 830 in FIG. 8. As described earlier, thecollector may be deployed through a tap 902. Alternatively, a switch inthe network can be configured to replicate all traffic to a specificport on the switch. As a result, it can see all the traffic coming fromthe network. In another embodiment, the monitoring platform can get thesession-level information from an existing mobile router such as a GGSNor a PDSN. In another embodiment, this information can be obtained fromserver logs from a content provider 906. The content session requestinformation is collected continuously as session requests arrive frommobile devices.

2. The Mobile Number module 920 continuously collects the mobile numberand IP address pair and saves it to the database 930 tagged by the IPaddress. Specifically, when a user's data session is established (PPP inCDMA or PDP context in UMTS), the entity creating the session requestsan IP address. For instance, in CDMA, when the PDSN is creating a Pointto Point Protocol (PPP) context, it queries the AAA for an IP address.Similarly, in UMTS, when a SGSN requests a Packet Data Protocol (PDP)session with the GGSN, the GGSN assigns an IP address. A similarapproach may be used for other mobile networks such as Wimax. ThePDSN/HA or GGSN in turn passes this IP address information via RADIUS toother servers. The monitoring platform monitors these requests to getthe IP address-mobile number pair. More details are provided inconnection of the description of FIG. 10. In another embodiment thisinformation can be retrieved from a AAA database using a LDAP or RADIUSprotocol, if it is designed to store the mobile number request. Thisdepends on operator policies. In another embodiment this information canbe obtained by the monitoring platform being a RADIUS proxy in betweenthe GGSN and AAA or PDSN and AAA. This can then get access to the mobilenumber and IP address being requested. In another embodiment, thisinformation can be collected by a transparent tap on the RADIUS serveras well, to collect this information. The mobile number information iscontinuously collected as new session initiation requests are made frommobile devices. The monitoring platform tracks the IP address and thecorresponding Mobile Number in its internal database.

3. The Location module 960 continuously collects the locationinformation and maps it to a physical location for each new IP addressdetected coming on line. This information is retrieved using the mobilenumber and saved by the IP address. In one embodiment this informationcan be collected once every time a new session is created. In this case,the HLR or LBS platform is queried when a PDP/PPP context request iscreated. In another embodiment this information can be regularly polledat a configurable frequency from the HLR/LBS platforms 961. In anotherembodiment, if the platform allows pushing, this information can bepushed to the Monitoring Platform. If the information is to be collectedfrom the HLR, typical MAP requests such as MAP_SEND_AUTHENTICATION_INFOand the corresponding response MAP_INSERT_SUBSCRIBER_DATA can be usedfor UMTS based networks. In CDMA based networks, messages such asMAP_QUALREQ may be used. If the information is collected from the LBSplatform 961, protocols such as Location Interface (LIF), Lg interface,or other specific APIs may be used. This information is stored in theinternal database against the user's IP address entry. The locationinformation obtained from the LBS platform further needs to be mappedinto information useful to the content providers, such as a zip code,postal address, or marketing area. This mapping can be done through asecond dip into a geocoded database 963 or 965 for thelatitude/longitude data. Similarly mapping a cell sector to a region canbe done by using the mobile operator's information.

4. The Demographics collection module 940 continuously collects theuser's demographic information for each new IP address/mobile numberdetected in the system. This information is captured from the carrierdatabases using the mobile number and stored into the database indexedby the IP address. This information can be cached for a configurableperiod and refreshes can be obtained beyond that period. Example sourcesof data include a subscriber database 942, HLR 946, CRM database 944,AAA database 942, etc. The information is collected from the CustomerRelation Management (CRM) database 944 or through a LightweightDirectory Access Protocol (LDAP) database 942 or through the HLR 946. Inthis case, protocols such as XML, LDAP, or Mobile Application Part (MAP)may be used. The retrieved information is segmented into categoriesbased on age, gender, income, etc. and a single category identifier maybe stored internally to allow anonomization. This information istypically collected every time a new mobile number is received. Examplesof categories include: {[18-25 yr, male], [18-25 yr, female], [25-35 yrmale], [25-35 yr, female], [35-55 yr, male, middle income], [35-55 yr,female, middle income], [over 55 male], [over 55 female]}. Otherdimensions such as education, occupation, can be added.

Other information such as the device type, network speed, network typeetc. can be obtained from the information already collected. Forinstance, the content request contains the User Agent information, whichgives details on the device. The network type can be obtained from theother HLR data retrieved.

These four processes (910, 920, 940, and 960) run independently on thecollector device and the data collection module collects and correlatesthe data and saves it into the database. The information from theseprocesses is preferably collected prior to module 910 collecting sessioninformation. All the information is ready when the content request comesin, thereby making it possible to send the information in real-time.

In another implementation, the different modules could write thecollected information to an internal database, either independent orshared. The Collection and Correlation Module can put the informationtogether and write the result to the database.

In another implementation, the mobile number, location, and demographicsinformation can be written to a database as collected. The real-timesession information is sent to the Data Collection and CorrelationModule, which correlates it with existing data to generate a real-timestream.

The overall IP address assignment and mobile number or mobile identitynumber (also referred to as MIN) association is described in thefollowing FIG. 10. As shown in the figure, the IP address is assigned bythe GGSN/PDSN/HA 1010 and remains active for the duration of thesession. The IP address is reassigned based on an inactivity timer,which is vendor- and operator-specific. Each session gets assigned a newIP address. The IP number to MIN mapping is sent to the RADIUS serverthrough the START 1020 and STOP RADIUS packets 1030. The table 1040shows how the collector can maintain a mapping between IP addresses andmobile numbers.

The mobile number can be obtained by the collector in one of severalways.

1. In general, when the PDP or PPP session is established, the GGSN/PDSNsends a RADIUS start/stop packet for accounting purposes. This messagecontains the MIN and possibly other user information. The collector cansniff the RADIUS traffic on the output of the GGSN/PDSN to get thisinformation. This is shown in FIG. 1-C.

2. In most networks the GGSN can be configured to send a RADIUSstart/stop packet with Vendor-Specific Attributes (VSAs) that captureuser information (e.g. MIN, SGSN address for this session, and possiblyother information.). This information can be sent to multiple recipientsin the mobile network and the collector can be configured to be one suchrecipient. The specific VSAs are operator dependent. This will requirespecific information from the operator.

3. In some networks, the GGSN tends to provide “Enriched” headers to thetraffic going into the “on-deck” destinations. This is usually operatorand gateway specific and would require additional integration within anoperator. This information may contain user ID, MIN, SGSN location, etc.This also works only for on-deck traffic, but could be a way to getadditional information.

As mentioned in Step 315 of FIG. 3, the captured data is filtered beforepassing on to the Data Manager. While images in general can be filtered,it is required to capture advertisements or ads. FIG. 11 is a signalingdiagram of layer 7 messages and shows how image advertisements can bedetected.

1. The browser requests the original page.

2. The content site requests an ad from an ad server.

3. The server returns a pointer to the ad.

4. The content site returns a http response to the client with the imagetag.

5. The client requests the link pointed to by the ad. By trackingaccesses going to 1110 the collector can identify that an ad is beingserved.

6. The client renders the page locally.

7. If the user clicks on the ad, then there is a http request to thelink pointed to by the ad. This allows the collector to track a clickthrough. The Referer information is used to identify the URL that the adwas placed on.

FIG. 12 shows how multiple collectors can be managed through acentralized console. Each collector may have an embedded Web server anda centralized console will provide access to manage the collectorparameters. This console allows configuring and monitoring eachcollector. Collector-specific parameters such as IP address, faults, andalarms may be monitored. Also, other parameters such as filters forwhich URL or which IP addresses to capture can also be specified throughthis interface. Further, the amount of data stored in a collector, thefrequency with which it is transferred, whether or not it is masked, areother configuration parameters.

FIG. 13 shows how the data from the collector is further passed on tothe Data Manager. Multiple collectors can feed the Data Manager 1310.The Data Manager comprises two key modules, the Data Processor 1320 andthe Database or Data warehouse 1330. The Data Processor is responsiblefor collecting the data from multiple collectors, augmenting it withadditional information, and storing it into the database per steps 335,340, and 350 in FIG. 3. The database is the core storage module thatholds the report and event data. The database is accessed from theReports Manager.

One of the characteristics of the monitoring platform is that itcombines both reporting and analytics within a single platform. Thiscore feature drives the architecture of the Data Manager. Specifically,the monitoring platform supports two types of reports.

1. Pre-defined reports: This is a set of reports that is pre-definedacross different dimensions. The data for all these reports is generatedthrough aggregation and rollups frequently. As a result, these reportsare available whenever the user accesses the system.

2. Ad-hoc reports: While the pre-defined reports are expected to cover90% of the use cases, the system needs to be flexible enough to supportany ad-hoc report. However, to optimize on the overall system cost, theamount of ‘detailed’ data is stored is limited to a specific interval.This allows the user to run any ad-hoc query in real-time. Since thisquery is on limited data, the system allows the user to add such anad-hoc report to its set of selected reports for subsequent usage. Theunderlying design allows such an ad-hoc report to become a pre-definedreport without requiring any manual customization.

FIG. 14 shows more details within the Data Manager.

The Data Manager is built on some system-specific schema, which isdescribed below.

1. System-level Dimensional Data Definition 1408: This is a system-leveldefinition of all the dimensions and attributes in the system. Forexample, dimensions include: Content, Device, Demographics, Location,etc. Each dimension may have additional attributes, for instance, deviceattributes include type, manufacturer, etc.

2. Collector Data Format 1402: This defines the format for the data sentfrom the collectors to the DM.

3. Event Data Format 1404: This defines the schema for storing thedetailed event data within the warehouse. The ETL process in the DM(called Data Processor) converts the data from the collector DataDefinition format to the Event Data format, adding additional fields asrequired.

4. Pre-defined Report Data 1406: This contains the data that describesthe pre-defined reports in the system. Specifically, it includes thedefinition of the pre-defined tables in the Data warehouse as well as alist of the procedures available to update and query the tables forgenerating the pre-defined reports. Note that as new reports are addedto the system from new adhoc queries, this pre-defined report data getsupdated.

5. List of pre-defined reports 1409: The list of pre-defined reportsspecified in the Data Manager is also tracked in the Reports Manager.The Reports Manager builds this list by synching the data from thePre-defined Report Data in the Data Manager. This list includes thedefinition of each report as well as the query procedures used to buildthe data for that report.

The Data Manager comprises 4 key modules: Data Processor 1410, Datawarehouse 1420, Analytics Module 1430, and Ad-hoc Reports Generator1440.

1. Data Processor 1410: This module is responsible for receiving datafrom collectors (step 1412), correlating it with any additional data(step 1414), inferring other information from this data, and storing itinto the Data Store in the appropriate schema for the detailed data(step 1416). This module has 3 functional components:

-   -   1) Receive data 1412: This is the process that gets data from        different collectors. The data format for this data is defined        as the collector Data Format;    -   2) Correlation and Enrichment 1414: This process takes each        event and augments it with additional information. For example,        it could add other information to the data such as demographic        information obtained from a CRM feed at the DM. In this case, it        would take the collector event, see the phone number, and add        the demographic information for this phone number to augment the        information obtained from the collector. Additionally, it could        infer other information on the data. For instance, the collector        data would specify the URL and the Data Processor would then map        it to a specific category based on a definition file (e.g.        Entertainment). The result of this step is that each Event from        the collector is characterized by all the parameters and the        ‘enriched’ data is now ready to be loaded into the database. In        addition, the process could add additional metrics to enrich the        data. Typical metrics would include data consumed, time spent,        etc.;    -   3) The Load process 1416 loads the Event data into the data        warehouse per the schema, which is defined in the Event Data        Format.

2. Data warehouse 1420: This is the core database, which manages thestorage of the data. This stores the data in two forms: (a) basic‘event’ data (detailed data) 1422, (b) aggregate tables that are definedbased on the pre-defined reports 1424. The Data warehouse is queried bythe Reports manager through a standard interface. The Event Data Formatstores the description of the data warehouse schema for the Event dataand the Pre-defined Reports Data file stores information about aggregatetables corresponding to the pre-defined reports.

3. Analytics Module 1430: This module comprises the procedures thatmanage the update and query of the different tables:

-   -   1) Update Procedures 1432: Specifically, the detailed event data        is used as the ‘scratch pad’ or ‘staging area’ for updating the        aggregate tables. The update procedures will either be called        from within the database periodically as the event data is        loaded into the database or they may be called by the Data        Processor. If a new table is defined as part of scheduling a new        report, a new update procedure may be programmatically generated        and added to this list of procedures;    -   2) Query Procedures 1434: This includes the procedures used to        query the data. These include both the pre-defined procedures        for the reports available as part of the base system as well as        any new procedures that are created as part of scheduling new        reports.

4. Ad-hoc Reports Generator: This module supports the programmaticgeneration of new tables and aggregates based on requests from theReports Manager to generate new reports based on ad-hoc queries:

-   -   1) The Commands Processor 1442 parses the request from the        Reports Manager for creation of ‘new’ reports;    -   2) The Schema Manager 1444 generates new tables based on the        request. This process also updates the Pre-defined Report Data        with information about the new table. The Schema manager knows        to create a new table by comparing the dimensions and metrics        required in the query with the dimensions and metrics available        in the existing tables.    -   3) The Query Generator 1446 creates new queries to retrieve data        from the newly created tables. These queries are then added to        report-query list managed by the Analytics Processor;    -   4) The Update Generator 1448 creates new procedures to generate        aggregates for the newly requested report.

FIG. 15 shows further details on the Data Processor. The operation ofthe DP is as follows:

-   -   1. The seed data for all the dimensions is provided to the        database in step 1550;    -   2. The dimension mapping process 1530 retrieves the dimensions        from the database to get access to the indexes as specified by        the database;    -   3. The step 1512 receives data from multiple collectors;    -   4. The step 1514 receives data from other sources such as server        logs, client feeds, location platforms, demographics databases,        etc.;    -   5. The step 1516 correlates the information collected in 1512        and 1514 and maps it to the indexes obtained through step 1530;    -   6. The step 1518 computes other metrics on this data, such as        time spent, data consumed, visits, etc. This requires managing a        state machine to be able to correlate sessions to compute the        metrics; and    -   7. The next step 1520 transfers the processed ‘event’ data to        the database.

FIG. 16 shows a typical schema for managing the data within theDatabase. The data is stored as a star schema, where the events arestored in the fact table 1610 and the dimensions are stored in separatedimension tables such as 1620. This storage optimizes space since eachdimension need not be spelled out in each event—only the index of thedimension needs to be used in the event table.

In addition to storing the event data, the Database also storesaggregate data through tables for combinations of dimensions. Forexample, tables include: [location, device], [location, demographics],[location, application], etc. for each combination, data is stored indifferent time boundaries. These aggregate tables are updated as newdata comes into the system. Based on these aggregate tables, the systemalso has a set of ‘top N’ tables that contain data for the pre-definedreports in the system. These tables are constantly updated. As a result,when the user queries the pre-defined report, the data is alreadyavailable. Depending on the storage available, different levels ofcombinations of dimensions may be pre-stored in the system. Forinstance, for 5 dimensions, a total of 10 combinations of 2 dimensionsis possible ({destination, device}, {destination, location},{destination, demographics}, {destination, application}, {device,location}, {device, demographics}, {device, application}, {location,demographics}, {location, application}, {demographics, application}. If3 dimensions were to be combined, in addition to the above 10,additional combinations include: {destination, device, location},{destination, device, application}, {destination, device, demographics},{destination, location, demographics}, {destination, location,application}, {destination, demographics, application}, {device,location, demographics}, {device, location, application}, {location,device, application}, {location, device, demographics}, {location,demographics, application}. Each table is identified with thedimensions. When a query is issued, the list of tables is checkedagainst the dimensions in the query to check whether a pre-defined table(and hence report) exists for that query.

The update of different tables is explained with the help of thisexample:

To explain the concept further, consider the following example:

-   -   Events from the DP would be like:    -   1. Source IP1, Dest IP2, phone number 1, demographics=18-24m,        location=boston, device=iphone    -   2. Source IP2, Dest IP2, phone number 3, demographics=18-34,        location=Kansas, device=blackberry    -   . . .    -   These events are stored in the Event table in the database    -   At the same time, one of the aggregate tables [location,        demographics] lists combinations of demographics and location.        Some of the entries in this table include:    -   Loc=boston, demo=18-24    -   Loc=boston, demo=25-35    -   Loc=Kansas, demo=18-24    -   Loc=kansas, demo=25-35    -   As part of the table update process, the aggregate loading        process will increment the count for Loc=boston, demo=18-24—    -   related to first event and loc=Kansas, demo=18-24 for second        event    -   Then if the Reports manager queries for breakdown of boston by        demographics or breakdown of Kansas by demographics, the        existing aggregate tables are queried    -   If the Reports Manager queries for breakdown by loc=boston,        demo=18-24, and device=iphone, there is no aggregate table. In        this case, the event table is queried for that data. If the user        selects to add this report to the predefined reports, then a new        aggregate table of combination [location, demo, device] is        constructed, where some of the entries are:    -   Loc=boston, demo=18-24, device-iphone    -   Loc=boston, demo=25-34, device=iphone    -   Loc=Kansas, demo=18-24, device=iphone    -   Loc=Kansas, demo=25-34, device=iphone    -   Loc=boston, demo=18-24, device=blackberry    -   Loc=boston, demo=25-34, device=blackberry    -   Loc=Kansas, demo=18-24, device=blackberry    -   Loc=Kansas, demo=25-34, device=blackberry

Note that once this table is created programmatically, it is updatedperiodically along with other aggregate tables. As a result, this newreport is also now a pre-defined report and available whenever thesystem is queried.

Metrics such as time spent, data consumed are computed in the dataprocessor module. Unique visitor count is a metric that is of interestas well. This metric is computed within the database. Specifically,themobile number is used to identify unique visitors. This metric iscomputed from the data in the event table for a specified interval. Forexample, if the system needs to query the number of unique visitors to adestination such as cnn.com within the last week, a query would be runon the event table data for the last week against cnn.com and the numberof unique phone numbers is determined to get a count for uniquevisitors.

FIG. 17 show the details of the Reports Manager. The database interface1720 interfaces with the database to retrieve results from queries. The1730 is the main server within the Reports Manager that generates theuser interface. Multiple end-users connect to the Reports Manager toview the reports.

The operational flow for the system is summarized in the following FIG.18.

1. In step 1810, the User of the Reports Manager selects a report bynavigating through the UI. The user could do one of three things:

-   -   a. Select a pre-defined report    -   b. Create an ad-hoc query    -   c. Schedule a New Report based on a specific ad-hoc query.

In all cases, the request is internally mapped by the Reports Managerinto a specific query. The query is categorized by dimensions andattributes.

2. In step 1820, the Reports Manager checks whether a pre-defined reportfor this set of dimensions and attributes already exists by checking thelist of pre-defined reports. If the report exists,

In case the report is a Pre-defined report, as in step 1830, the reportsmanager does the following functions:

-   -   1. If it is a pre-defined report, the appropriate query is        retrieved from the report definition file;    -   2. The Reports Manager issues the query to the Data warehouse in        the Data Manager;    -   3. Data Manager retrieves the data; and    -   4. Reports Manager displays the report.

In case the report is an Ad-hoc report, as in step 1840, the reportsmanager does the following steps:

-   -   1. If the report is an ad-hoc query, the Reports Manager        constructs the query and sends it to the Data warehouse in the        Data Manager;    -   2. The data warehouse runs the query on the detailed event data;    -   3. The Data Manager generates the result, and sends the data to        the Reports Manager; and    -   4. The Reports Manager selects the right type of report based on        the report type and constructs the appropriate report for        display.

In case the report is a new pre-defined report, as in step 1850, thereports manager does the following steps:

-   -   1. If the user requests to schedule a new report based on a past        ad-hoc query, the system needs to create a new ‘pre-defined’        report. In this case, the Reports Manager sends the command to        the Ad-hoc Report Generator module in the Data Manager;    -   2. The Data Manager receives the request; and    -   3. This module parses the request (Commands Processor), creates        a new table if required (Schema Manager), creates a new query to        generate the report (Query Generator), and creates a new        procedure to periodically generate the aggregate data (Update        Generator). Based on this, the internal report-query list is        updated. From this point on, the new report is available as part        of the pre-defined reports in the system. Details of this are        described in the next section.

FIG. 19 shows the details of the ‘report selection and generation’module 1730 from FIG. 17. The Reports Manager data includes 1920, whichis the list of dimensions and metrics used in the system. Examples ofdimensions and metrics were provided in the context of the descriptionof FIG. 2, 1930 which is the list of pre-defined tables. These tablesare typically combinations of different dimensions. 1940, is the list ofexisting procedures in the Data Manager. For each pre-defined report,1940 lists the procedure to invoke in the database to get the result.For each pre-defined report in the system there is hence a pre-definedprocedure.

When the user accesses the system, the user client issues a request tosee a report. The request can be one of three types: pre-defined (A),new ad hoc (B) or a save adhoc (C). A request is of type A if the userclicks on a link for a pre-defined report. A request is of type B if theuser constructs a new query. A request is of type C if the userconstructs a new query and explicitly asks to ‘save’ this reports. Therequests are all passed from the user interface to the report generationlogic through a query language. The overall sequence within the module1730 is as follows:

-   -   1. If the request is a pre-defined report of type A, then the        reports manager gets the corresponding procedure from the list        1940 and invokes the appropriate procedure. The database runs        the procedure and returns the result;    -   2. If the request is an ad-hoc query of type B, the module 1730        generates a SQL query programmatically from the query language.        This query is passed on to the database, which returns the        result; and    -   3. If the request if of type C, where the user requests an        ad-hoc query to be saved for future as a pre-defined reports,        then module 1730 checks the data definition 1930 to see if a        table already exists within the database for the requested        dimensions    -   1. If a table exists, then 1730 maps the previous query against        that table, creates a procedure to reflect this query, and        stores the query within the database    -   2. If a table does not exist, then a new table with the required        dimensions, then 1730 creates a new table, instantiates it        against the database, creates procedures to load and update that        table, creates procedures to query that table. Tables 1930 and        1940 are the updated appropriately. As a result, the new ad-hoc        report now becomes part of the pre-defined reports in the        system.

FIG. 20 shows the overall structure of the user interface. 2010 showsthe pre-defined reports that are defined in the system. 2020 shows theability to look at different metrics within a report. 2030 shows theability to perform an ad-hoc query.

FIG. 21 shows how an individual report can be drilled down to multiplelevels. 2110 shows an example of a top-level query. The report could forinstance show the top ten destination URLs along with a tabulardescription 2112. The Reports Manager looks at the dimension the user isalready looking and appropriately presents other dimensions fordrilldown. For instance, if the user is viewing the destinationdimension, the available drilldowns could be at different attributeswithin the destination dimension (e.g. on-deck vs. off-deck) or bylocation, demographics, etc. The user can select a specific URL 2114 andclick on it to get more details. The detail allows the user to selectwhich dimension to drill down on—e.g. device type, content category,location, etc. This shows a drill down report 2120. The user may furtherselect an entity on this such as 2122 to see the next level of drilldown 2130. For any report, the user can also perform actions 2116 suchas printing, emailing, etc.

FIG. 22 shows how the invention can also be used to capture user-levelinformation in a representative form. To support user level information,an additional table needs to be created in the Data manager. This tablewould list the set of users and their current data.

In step 2210, the list of active users in the system can be displayed. Aspecific user may be selected from this list. The event table would bequeried to check for the list of users. For the specific user, therelated information is queried from the Event table. In addition, it isalso possible to store historic data about a user.

In step 2220, for a selected user, further details related to URL,content type, demographics, location, device, and network can beobtained by clicking the appropriate link.

In Step 2230, by clicking the URL, the history of the URLs accessed bythis user for this session may be displayed. Other details as the timespent on each URL and the category can be shown.

In step 2240, by clicking the content type, the type of application canbe displayed.

In step 2250, the details of the demographic of this user may be shown,including the segment, rate plan, etc.

In step 2260, the user's current location may be displayed.

In step 2270, the capabilities of the device used in this transactionare shown.

In step 2280, network level information, including data consumed detailsmay be shown.

In general, correlation can be done in several ways. In Option 1, thecorrelation can be done at reporting time. In Option 2, the correlationcan be accomplished by creating new entries or a new database withcorrelated information as and when data is collected.

In addition to storing the event data, the Database also storesaggregate data through tables for combinations of dimensions. FIG. 23shows the details of how aggregate tables and defined reports aredesigned into the Data Manager.

The Data Manager consists of aggregate tables for different dimensionsand combinations of dimensions. An aggregate table is defined as a tablethat contains aggregate information, aggregated over users and timeintervals. As shown in FIG. 23, 2320 and 2330 represent two tablescomprising different dimensional combinations. 2320 could be acombination of destination and device, while 2330 could be a combinationof destination, device, and location. Further, each dimensionalcombination needs to be tracked for different time intervals. As shownin FIG. 23, 2320 tracks table 1 for an hourly time interval, while FIG.2322 tracks the same table for a daily interval. This means that allevents that occur at a time stamp within an hourly interval areaggregated in 2320. 2322 tracks the data over all the hours in the day.As a result, depending on the desired time interval for a report,different tables can be queried.

For example, the event data coming from the collectors comprisesuser-level data that includes the user's identity and a specific timefor the event:

Timestamp1, Source IP1, phone number 2, destination url=cnn,device=blackberry, location=boston, etc.

This information may be populated into an aggregate table for thecorresponding dimensions within the time interval. The timestamp anduser-level information is removed in this case.

If there are N dimensions, then there are a total of N+NC2+NC3+ . . .+NC (N−1) such dimensional combinations. Storing data for all suchcombinations is expensive in terms of storage as well as time to update.The Data Manager is designed to have aggregate tables for a specific setof dimensions, based on the types of pre-defined reports and alloweddrill downs. The system maintains a list of existing dimensionalcombinations available, as shown in 2310.

Reports are designed to report on a specific time interval and acrossusers, e.g. show me number of 18-34 male users coming from boston in thelast hour. Keeping this in mind, the system is designed to deliver allreports without having to store every detailed event data. In this case,the aggregate tables need to only store a combination of location anddemographics dimension for different time intervals. Further, theaggregate table may have all the possible combinations of demographicsand locations. However, pre-defined reports may be interested only in asubset of locations and demographics, in this case it is 18-34 male andBoston. To accommodate this optimization, the data manager also managespre-defined report tables 2340. The specific pre-defined tables thencorrespond to the specific report. 2342 for instance, may be a tablestoring data for users from key locations and corresponding distributionby demographics. 2344 may be a different report, showing distribution oflocations by application type.

When the reports manager queries for a specific report, the query ischaracterized by the dimensions involved. The system first looks at thepre-defined reports to see if a report already exists for that query. Ifthat report doesn't exist, and if the aggregate table exists, then a newreport table is created. If the combination of dimensions doesn't exist,then a new aggregate table has to be created and the list of tables 2310needs to be updated. The list of existing reports 2350 should also beupdated to add the new report to this list.

To explain the concept further, consider the following example:

Eevents from the DP would be like:

-   1. Timestamp1, Source IP1, Dest IP2, phone number 1, demographics    =18-24m, location=boston, device=iphone-   2. Timestamp2, Source IP2, Dest IP2, phone number 3,    demographics=18-34, location=Kansas, device=blackberry

These events are stored in the detailed event data table in thedatabase. At the same time, one of the aggregate tables [location,demographics] lists combinations of demographics and location. Thehourly table is updated with these events. Some of the entries in thistable include:

-   -   Loc=boston, demo=18-24    -   Loc=boston, demo=25-35    -   Loc=Kansas, demo=18-24    -   Loc=kansas, demo=25-35

As part of the table update process, the aggregate loading process willincrement the count for Loc=boston, demo=18-24—related to first eventand loc=Kansas, demo=18-24 for second event for the appropriate timeinterval.

A pre-defined report would be ‘for specified locations, show adistribution by demographics’. In this case, a pre-defined report tablewould cover the locations, and for each location, distribution bydemographics.

Then if the Reports manager queries for breakdown of Boston users bydemographics or breakdown of Kansas users by demographics, the existingpre-defined tables are queried and no new report needs to be created.

Now if the user queries for distribution of users for a location bydemographics for a location not in the pre-defined report table, thesystem would go back to the appropriate aggregate table to get the datafor location L. If the user so requires, a new Report table 2346 forsuch report is instantiated. Note that in this case since the dataalready existed, a new aggregate table was not required to be created.

Now suppose the user queries for breakdown by loc=boston, demo=18-24,and device=iphone, there is no report table. Further, the system is notcollecting data for a combination of location, device, and demographicsand so no aggregate table exists. In this case, the detailed event datatable is queried for that data. Since the event table has the detailedevent data, the results for this data can be presented to the user. Ifthe user selects to add this report to the predefined reports, then anew aggregate table of combination [location, demo, device] isconstructed, e.g. 2360, where some of the entries are:

-   -   Loc=boston, demo=18-24, device=iphone    -   Loc=boston, demo=25-34, device=iphone    -   Loc=Kansas, demo=18-24, device=iphone    -   Loc=Kansas, demo=25-34, device=iphone    -   Loc=boston, demo=18-24, device=blackberry    -   Loc=boston, demo=25-34, device=blackberry    -   Loc=Kansas, demo=18-24, device=blackberry    -   Loc=Kansas, demo=25-34, device=blackberry

By creating the new aggregate table, the data for the combination oflocation, demographics, and devices starts getting automatically updatedinto the system. A corresponding pre-defined report 2348 is also createdinto the system

This user level information can be stored in an internal database toderive user profiles that can be used for targeting and monitoring.

Preferred embodiments of the invention help service providers answer anumber of key questions for mobile operators:

1. Applications: What kind of instantaneous traffic is happening in mynetwork—how many users are doing video, p2p, audio, web, games? How isthe traffic distributed across different applications—what fraction ofusers are using video, what fraction of bandwidth is used by video?

2. Application Details: For a specific application such as video, whatprototols are being used—HTTP streaming, RTSP streaming, Flashstreaming? For a specific application such as video, what applicationsare users using to use these applications—Flash, RealPlayer,MediaPlayer, QuickTime, iphone youtube client, custom clients, etc. Whatare the popular devices and applications on my network?

3. Content Sources: What are the top destinations users are going to inorder to get their content? Should I strike special partnerships withthese popular content providers?

4. Content Consumption by Location: Where is this content being consumedfrom? What are the top 20 locations (regions, states, cell sites) thatare consuming this content? Are there regions that are accessing morecontent and hence need to have either more capacity or more optimizationin the network?

5. Content Consumption at Content provider level: For a specific contentprovider, what is the distribution across locations? These high-usagedestinations make for good targets for local advertising.

6. Content Distribution and Repeatability: How cacheable is thiscontent—is there a long tail or not? What fraction of this content isrepeat content? What is the bandwidth consumed by the repeat content asa fraction of total bandwidth? Can I save on bandwidth by caching orhosting this content locally?

7. Content Rates and Duration: What content rates are people consumingbandwidth at? What is the typical duration of content accessed? How muchis there a need for sustained long duration high speed content vs.bursty content? How should I optimize the network to support thiscontent mix?

8. Content Consumption by Demographics: For each demographic group (byage, gender, income, home location), what are the top contentdestinations? For each demographic group, what are the top locations forhigh usage? This will help me figure out targeting advertisements.

9. Content Classification: What is the distribution of types of contentaccessed by users—sports, media, news? What is the split between usergenerated content vs. traditional studio generated content? For aspecific content provider, what is the distribution of types of content?

10. Time of day distribution: Do any of the above patterns change basedon time of day?

Preferred embodiments of the invention also help content providersanswer a number of key questions:

1. Mobile usage Patterns: How much content consumed by mobile users? Howmany streams going to mobile? What is the percentage growth? How longare users staying? Information on repeat users?

2. User location: Where is my mobile audience? What are the top 20geographical locations my users are coming from at any time? What is thedistribution?

3. Content classification: Location-level information for each contenttype?

4. Devices and Network Speeds: What phones are users using? What are thecapabilities of devices? What rates are users coming in? How do I needto tailor my content for best viewing?

5. Bandwidth consumption: How much bandwidth is being consumed by mobileusers? What percentage of content is repeat content? How will I benefitby caching to reduce my CDN/Hosting bill?

6. Mobility: Is my user base mobile? What percentage of users areconsuming content in one location vs. in mobile locations?

7. Demographics: What is the distribution of mobile content usage basedon demographics?

8. Competitive: How does consumption for a specific content providercompare with another competitor's consumption?

Under some embodiments, a method, involves a non-inline approach tocollect data off the network and from other sources, filter relevantevents, for selected events correlate the mobile and IP identities,enrich selected events with other metrics, store data within thedatabase, and compute reports in-line as data is stored.

Under some embodiments, the design allows deploying the solution withoutrequiring changes to the network, applications, or devices.

Under some embodiments, the design allows data to be collected fromsoftware clients on end-user devices.

Under some embodiments, the design allows correlating data acrossmultiple dimensions characterizing the user, including application type,content type, demographics, location, device type, network type, etc.

Under some embodiments, the logic allows a user to query the system forreal-time reports, where the report data is provided through a datastructure that manages multiple aggregates without requiring storage ofthe entire set of events.

Under some embodiments, user-level information can be obtained from thesystem.

Under some embodiments, the logic allows drill downs of the reportsacross different dimensions and via different metrics through the abovementioned data structure.

Under some embodiments, the design allows ad hoc queries to the datausing a graphical language, and conversion of selected ad-hoc queries tosaved reports for subsequent pre-computation.

Under some embodiments, the design allows the data produced by theinvention to be used by service providers.

Under some embodiments, the design allows the data produced by theinvention to be used by 3^(rd) party content providers and advertisers.

Under some embodiments, the design allows the data produced by theinvention to be used by a service provider to optimize the delivery ofmobile content within their network through a mobile-aware caching andcontent delivery system.

Preferred embodiments of the invention provide a monitoring platformthat provides non-intrusive, transparent, real-time collection of dataapplication-level activity and other user information on a mobile datanetwork, and subsequent multi-dimensional reporting and analysis of thedata. In order to effectively mine a carrier's network to collect mobiledata-usage information and provide it to an operator, there are threemajor steps: data collection, data storage, and operator interface.

In the data collection phase, collection devices are placed incommunication with routers in a mobile data network. By collecting thedata in this manner, the collectors are able to non-intrusively tap intomobile-data streams and obtain data in real-time.

Once data has been collected from the network, the data collectorsfilter the collected data so that just application-level data remains.The data collectors also obtain mobile phone numbers from either an AAAdatabase or by tapping into a RADIUS server, and correlate these numbersto their corresponding IP addresses. The collectors then send thefiltered data and mobile phone number-IP address pairs to a storageunit.

A data storage unit receives the collected data, and further augmentsthe collected data with supplemental information, such as demographicinformation, mined from outside sources, such as logs, client-generateddata, and SMSC databases. The data is then stored into a database asdetailed event data for a predetermined period of time. By storing allof the detailed data, the monitoring platform is able to produce datarequested for any ad-hoc query, as long as the data requested wascollected within the predetermined period of storage.

The storage unit populates aggregate tables, which have defineddimensions, with newly collected data matching these dimensions. Bycontinuously populating the aggregate tables with collected data, themonitoring platform is able to produce data requested for definedreports in real-time.

An operator also has the option of converting an ad-hoc query into anew, defined report. By transforming the query into a defined report,the operator can ensure that the requested report will reflect all offthe relevant data collected from that point forward.

An operator interface acts as the link between the operator and thestored information. The interface determines whether the operator hasrequested a defined report, ad-hoc report, or new defined report, andretrieves the appropriate data from the data storage in real-time.

After displaying the report, the interface allows the operator tofurther analyze the report by drilling down into the data by a pluralityof different dimensions. The interface reports the dimensional analysisin real-time, and then gives the operator a new batch of dimensionsrelated to the previous drilled dimension with which to conduct furtheranalysis. This approach allows the monitoring platform to give theoperator a multi-dimensional analysis of the collected data inreal-time.

While the specific details are provided for operating this system in amobile network, the approach is in no way limited to a mobile network.The same methodology can be applied to other networks, includingbroadband cable, DSL, WiMAX, and other networks. Equivalent informationcan be extracted from similar sources of data and similar analytics canbe applied to mine the collected data.

While the above describes a particular order of operations performed bya given embodiment of the invention, it should be understood that suchorder is exemplary, as alternative embodiments may perform theoperations in a different order, combine certain operations, overlapcertain operations, or the like. References in the specification to agiven embodiment indicate that the embodiment described may include aparticular feature, structure, or characteristic, but every embodimentmay not necessarily include the particular feature, structure, orcharacteristic.

While the present invention has been described in the context of amethod or process, the present invention also relates to apparatus forperforming the operations herein. This apparatus may be speciallyconstructed for the required purposes, or it may comprise ageneral-purpose computer selectively activated or reconfigured by acomputer program stored in the computer. Such a computer program may bestored in a computer readable storage medium including, withoutlimitation, any type of disk including optical disks, CD-ROMs, andmagnetic-optical disks, read-only memory (ROM), random access memory(RAM), magnetic or optical cards, or any type of media suitable forstoring electronic instructions.

While given components of the system have been described separately, oneof ordinary skill also will appreciate that some of the functions may becombined or shared in given instructions, program sequences, codeportions, and the like.

1. A data manager device for receiving information from multiplecollector devices about application-level data activity on said mobiledata network so as to provide dynamic real-time reporting about usage ofsaid mobile data network, said data manager device comprising: adatabase for storing received application-level data for a predeterminedamount of time; a data processing module for receiving information fromsaid multiple collector devices and storing said received data in saiddatabase, said data processing module determining which of said receiveddata corresponds to a set of defined reports and storing saidcorresponding data into data structures stored on said data manager,wherein said data correspond to said set of defined reports, eachdefined report including a combination of one or more of demographicinformation about users of said mobile data network, informationidentifying destinations for user data requests on said mobile datanetwork, and estimated physical location information about users of saidmobile data network, wherein data that is determined to correspond todefined reports is used to populate said reports in real time forviewing by an operator; and an analytics module for accessing saiddatabase to satisfy operator queries about usage of said mobile datanetwork, said analytics module being responsive to operator commands tomodify said set of defined reports to incorporate an operator query as anewly-generated defined report so that subsequently said data processingmodule populates said new report in real time.
 2. The data managerdevice of claim 1 wherein each report has a corresponding table whichspecifies the data to be automatically retained and continuously updatedby said data manager to populate a report.
 3. The data manager device ofclaim 1 wherein said reports specify aggregate data about all or asubset of users of said network, said aggregate reports including atleast a subset combination of the following set of aggregate date:physical location of users; demographic information about users; contentsite accessed by users; media type used by users; device type used byusers; network type used by users.
 4. The data manager device of claim 1wherein said information about application-level data activity includesIP-address-mobile-phone-number pairs for mobile devices using saidmobile data network, and wherein said data manager device uses saidphone numbers to query carrier information about the users to augmentthe received information with said carrier information or informationderived from said carrier information.
 5. The data manager device ofclaim 1 wherein said data manager determines session metrics about saidusers and at least one report uses said session metrics.
 6. The datamanager device of claim 5 wherein said session metrics include at leastone of page views, number of sessions, number of visits to content,unique visitors to content, time spent during sessions, data consumed bya user during a session.
 7. The data manage device of claim 2 wherein,in response to said operator command to modify said set of definedreports, said analytics module compares said operator query to saidtables to determine whether at least one table satisfies said operatorquery and if not said analytics module programmatically creates a newtable to satisfy said query and modifies the set of defined reports toinclude a new report corresponding to said new table.